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Abstract 

As states have become more active in estabhshing curriculum content 
standards and related assessments disappointingly httle attention has 
been paid to pohcy efforts that create learning opportunities for 
students to meet the new standards. This study examines one state 
pohcy designed to bolster the opportunity to learn by mandating 
additional instruction for students not currently achieving proficiency 
in the state standards. The results focus on a detailed description of 
New Y ork State's Academic Intervention Services, including its 
organizational and instructional elements (e.g., staffing, sch^uhng, 
student grouping, instructional strategies) across NY S schools. While 
the majority of states have established curriculum frameworks and 
linked them to assessment instruments, this experience in NY may be 
unigue for its coordinated emphasis on intervention services 
(academic and non-academic) hnked to rigorous learning and 
accountabihty standards. However, the caveats identified in this study 
promote a famihar sense of local discretion in the interpretation and 
implementation of state pohcy mandates. The analyses describe how 
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such practices vary by local district context, such as community wealth 
and geography and if those practices have eguity implications. The 
primary analyses draw on survey data from a stratified random sample 
of 764 teachers and principals from 125 school districts, and feature 
multi-variate methods with proper adjustment for the clustering of 
responses within districts (i.e., multiple teachers and administrators 
within each district). 


Introduction! 

As states have become more active in estabhshing curriculum content standards and 
related assessments (Skinner & Staresina, 2004), disappointingly httle attention has been paid 
to policy efforts that create learning opportunities for students to meet the new standards 
(Wilson, 2004). The identification and proliferation of state curricula and state assessment 
systems are notable, and yet by themselves do httle to ensure improved or additional learning 
opportunities for low achieving students. This study examines one state pohcy designed to 
bolster the opportunity to learn by mandating additional instruction for students not currently 
achieving proficiency in the state standards. In an attempt to ensure a bridge between the 
mandated curriculum content standards and multiple state examinations. New Y ork State 
implemented a policy in 2000 cahing for the provision of Academic Intervention Services 
(AIS) to eveiy student achieving at levels less than proficient. 

Academic intervention services (AIS) are services designed to help students achieve the 
learning standards in English language arts and mathematics in grades K-12 and social studies 
and science in grades 4-12. These services include two components: 1) additional instruction 
that supplements the general curriculum (regular classroom instruction); and/ or 2) student 
support services needed to address barriers to improved academic performance. (NY S 
Commissioner's Regulations, Part 100.1(g))2 

A primary focus of this study entails a detailed description of AIS programming including 
its organizational and instructional elements (e.g., staffing, scheduling, student grouping, 
instructional strategies). We also analyze how such practices vary by local district context, such 
as community wealth and geography, to assess the eguity imphcations of consistent or varied 
policy response. T o accomphsh these goals, we draw on survey and archival data from a 
stratified random sample of 764 teachers and principals from 125 school districts. The sample 
of districts was drawn from the universe of school districts with high schools and includes the 
largest urban districts in the state as well as a representative sample of suburban and rural 
districts (See T able 1). Analyses include bivariate and multi-variate methods with proper 


1 The authors would hke to thank the G oveming Board of the Education Finance Research 
Consortium of New Y ork State for their support of this research. A fuller version of the findings 
reported here are contained in a report published by the EFRC entitled "The Implementation of 
Academic Intervention Services (AIS) in NY S: Implications for School Organization and 
Instruction". In particular, this study benefited from the careful reading and thoughtful suggestions of 
D eborah Cunningham and J eanne Post of the New Y ork State D epartment of E ducation. 

2 Available at http:/ / www.emsc.nysed.gov/ partlOO/ pages/ 1002i.html 
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adjustment for the clustering of responses within districts (i.e., multiple teachers and 
administrators within each district). 


Context 


Background in N ew Y oik State 

Beginning in the fall of 2000, New Y ork State (NY S) school districts were reguired to 
implement Academic Intervention Services (AIS) that hnked under-performing students vrith 
additional resources to improve their performance. Regulations mandate a new enrichment 
approach to provide additional instruction to students in grades K-12 who needed extra time 
and help to meet state learning standards. This strategy replaces the earlier rehance on later- 
grade remedial instruction. State pohcy mandates included specific guidelines for AIS 
implementation. For example, regulations now reguire students to receive services vrithin one 
academic semester of identification. Ehgibihty for AIS services should be based upon multiple 
measures of student performance, including grades, test scores, and recommendations of 
teachers and parents. SED guidehnes also suggested strategies to vaiy the intensity of the 
intervention, including ideas about scheduling, duration, and the level of student-teacher 
individualization^ 

AIS policy and practice represents an attempt to promote policy coherence between 
curriculum, instructional, and assessment pohcies. AIS programming may be conceptualized as 
a bridge between the curriculum and assessment programs, an alternative and supplemental 
instructional program that targets poorly performing students. The AIS guidehnes encouraged 
schools to generate additional instruction activity to meet student deficits. The policy 
guidelines, however, did not reguire exphcit strategies to change instructional content or 
pedagogy, or for that matter, provide suggestions on how professional development could 
improve instructional services within an AIS plan. The specificity of the guidehnes was more 
focused on identification criteria, as described above. This measured approach between pohcy 
prescription and encouragement of local innovation coincided vrith the dramatic change in 
learning and graduation standards in New Y ork State. 

Specifically, the State Board of Regents adopted new Learning and G raduation 
standards for ah pubhc schools in New Y ork State, and began implementing the new standards 
in 2000. Briefly, the reform reguires all students seeking a high school diploma to earn a state- 
endorsed Regents D iploma. While in practice for more than 100 years, the Regents courses, 
exams, and diplomas are only now mandated for ah students. Heretofore, less than one- half of 
ah high school graduates earned a state-endorsed diploma, with the majority earning a diploma 
meeting local school district standards. 

Related Literature 

This study of AIS implementation is timely given the long history of school systems 
avoiding or reinterpreting pohcy prescriptions from state agencies (Tyack & Cuban, 1995; 


3 Kadamus, J. (2000). Q and A. Guidelines on Academic Intervention Services Implementation. New 
Y ork State E ducation D epartment. The University of the State of New Y ork, Office of Elementary, 
Middle, Secondary and Continuing Education (available at 
http:/ / www.emsc.nysed.gov/ docs/ AISQAweb.pdf). 
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Tyack & Tobin, 1994). As a framework for this study, we introduce the literature on policy 
coherence, policy adherence, and instructional capacity. These terms provide a lens with which 
to understand this investigation into the implementation of state AIS policy. As a policy 
designed to build instructional capacity to assist low-performing students in achieving state 
curriculum and accountability standards, we posit that AIS represents an attempt to promote 
policy coherence between curricular, instructional, and accountability policies. Moreover, AIS 
policy is designed to bolster capacity by fiat (i.e., state regulation), the result of which is to add 
instruction services for students most in need. 

The notion of policy coherence received much attention in the early 1990s and 
continues today (Fuhrman, 1993; Wilson, 2004b). Researchers, policymakers, and practitioners 
continually attempt to understand, explain, and alter the many ways in which schools function 
in light of the multiple and often contradictoiy policy messages (Chrispeels, 1997; McLaughlin 
& Talbert, 1993; Smith & 0 'D ay, 1990a; SpiUane & Jennings, 1997). One argument to gain 
support for increased coherence was that various and often disconnected strains of local, state, 
and federal policy left local educators with choices as to which policies to adopt, which to 
change, and which to ignore (Cuban, 1998; Tyack & Cuban, 1995). Hence, to generate 
consistent, eguitable, and widespread educational change, it was suggested that designing 
tighter coherence among policies within and between different policy levels is possible and 
desirable (Clune, 2001; Smith & 0 'D ay, 1990b). Wilson (2004a) argues for the integration of 
assessments, curriculum frameworks, and classroom instructional methods from the level of 
the classroom to the highest levels of state and policy. What would constitute evidence of 
policy coherence at the district level? A measure of coherence would be the degree to which 
individuals within and across districts adhere to the specific policy guidelines outlined by the 
State. Moss (2004) terms this coherence throu^-alignment 

While much effort has been devoted to ahgning local curriculum and instruction with 
state curriculum framework and assessment systems, others argue that such coherence is not 
without risk (Moss, 2004). Moss guestions the virtue of tightly ahgned local and state policies 
given the wide variety of local contexts, offering instead a coherence throu^-negotiation-of-meaning 
argument: 

N egotiation of meaning can result in alignment and alignment can encourage negotiation of 

meaning such that local actors come to 'own' the concepts provided in the assessment 

system, (p. 219) 

Thus an alternative conception of policy coherence may not only be the ahgnment of practice 
across levels of the system, but may be interpreted as teachers understanding, taking 
ownership, and adapting the state-promoted instructional, curricular, and accountability 
practices to local student need. It remains to be seen whether the interpretation of policies 
differ randomly or systematically by context (e.g., geography, wealth, performance). 

The literature on policy coherence and adherence points to several opportunities that 
could stimulate variation in local implementation of state policy. First, if the policy scripts are 
too loose, districts may elect to interpret the policy guidance in a wide variety of ways; Ms may 
result in potentially superficial responses or basic discontinuity in a child's academic program. 
Second, if the policy is grounded within a conceptual framework and ahgned with 
complementaiy pohcies, local interpretation may be more constrained. A third perspective 
emphasizes the role of organizational capacity in the implementation of state policy. 
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This study supports the now prevalent reform discourse suggesting that organizational 
capacity, which includes teacher's professional knowledge and skills, financial resources, and 
effective leadership at the district and building levels, is important for increasing the 
organization's ability for dehvering high-guality instruction (Corcoran & G oertz, 1995; 

D arhng- Hammond, 1993; G alvin, 2001; McD onnell & Elmore, 1987; 0 'D ay, G oertz, & 
Floden, 1995). The existing research on capacity tends to stress how pohcy may stimulate or 
introduce capacity- building activities at the building level. Hightower's (2002) case study of 
instructional reform within the San D iego City School D istrict is illustrative of this. Here, the 
research points to the reform of capacity- building efforts such as developing school principals 
as instructional leaders, the hiring of school based instructional leaders (peer coaches) and staff 
developers, and the redirection and alignment of state and federal funding streams. Beyond the 
building level though, very little is known about capacity-building activities across intermediaiy 
educational organizations (Massel & G oertz, 2002). Here, the guestion addresses the role and 
emphasis of state, regional, district, and building level perspectives. Massel and G oertz also 
argue that as policy makers attempt to ahgn state instructional policy to provide more coherent 
guidance for classroom teachers in support of ambitious learning outcomes, district response 
to these initiatives must be taken into account. Others such as Firestone (1989) emphasize that 
the measurement of "a will to change" at the district level, interacting with a district's capacity 
to change, leads to substantial change within the district. 

In terms of local capacity to meet the academic needs of all students, how do school 
districts use the additional regulatory and fiscal incentives associated with heightened 
educational standards? Researchers have documented a variety of strategies districts use to 
increase the instructional capacity of schools. Providing and controlling access to data, 
professional development, curriculum and instructional guidance, gualified staff, and fostering 
relationships with external agents and networks are some of the strategies used by districts 
(G alvin, 2001; Massell & G oertz, 2002; SpiUane, 1996). Our own work in New Y ork State 
documents a variety of capacity building activities that emerged or substantially changed with 
the heightened learning and graduation standards, including academic interventions, teacher 
professional development, staffing changes, and the availability of alternative student 
outcomes (e.g., dropout, G ED, alternative education programs) (Sipple, KiUeen, & Monk, 
2004). Additional recent work documents how a broad range of capacity building activities at 
the district level are associated with local community characteristics, in ways that appear to 
disenfranchise children in impoverished schools (Sipple & KiUeen, 2004). In this work we 
control for geographic, wealth, and spending factors among NY S districts, and find that 
educators in districts serving high concentrations of poor children are hkely to report increases 
in alternatives to a coUege preparatory diploma (the Regents diploma), including transfers to 
G ED and alternative education programs. 

Research Questions 

This article draws on new statewide survey data to answer three main guestions: 

A . Policy Coherence. How do AIS implementation practices (i.e., 
organization and instruction) vary across the state? 

B. Pohcy Adherence. How well do district level implementation practices 
match the AIS pohcy guidelines outlined by the state? 



Killeen & Sipple: M andating supplemental intervention services 


6 


C . Capacity to Adhere. How well do districts possess the capacity to meet 
the AIS pohcy guidelines? 

These questions fall short of asking whether academic intervention services improve 
student achievement to proficiency levels, whether the pohcy is cost effective, or even just. 
Viewed through a lens of pohcy evaluation, and until sufficient data exist to assess changes in 
performance, important questions now must focus on descriptive accounts of pohcy 
implementation, organizational and instructional responses, as well as coherence and 
adherence at the local level. 


Data and Methods 

Our analyses wih focus on AIS implementation strategies among NY S school districts 
as reported by school principals and their teachers. This description relies on our discussion of 
item response variation by geography, wealth, and position, as weh as district performance 
categories. Summary analyses also examine typical correlates of program implementation levels 
including student race and student Enghsh proficiency. 

Survey Development 

Over the course of day-long site visits to four school districts in the fall of 2002, we 
conducted 45 interviews with approximately 90 principals, regular education teachers, AIS 
teachers and guidance counselors. We had previously visited these districts on several 
occasions during the preceding two years as part of our attempt to carefuhy document the 
district responses to and implementation of the broader set of new state requirements. D uring 
the 2000-2001 school year we interviewed more than 120 educators and community leadens 
and report these findings elsewhere (Sipple, Killeen, & Monk, 2004). The more recent wave of 
interviews specifically t^eted the relationship between AIS and other programs for 
underperforming and at risk students, the identification of students for AIS, how AIS is 
delivered, the focus of AIS instruction, as weU as the role of nonacademic intervention services 
designed to alleviate obstacles to academic success. 

Based on the findings from the quahtative interviews, we designed principal and 
teacher survey instruments. These instruments were designed to collect information about the 
process by which students are selected (and terminated) for AIS services, the scheduling and 
staffing of AIS programming, and the degree of participation in AIS among students in 
Enghsh and mathematics classes. While many questions in the surveys were asked of ah 
respondents (principals and teachers), teachers were asked to provide additional information 
about their actual classroom instruction, organization, and planning, while principals were 
asked to provide information about broader school-wide organization, scheduhng, and pohcy 
issues. Surveys ranged from 12-43 minutes in length and were conducted by trained staff using 
a computer-assisted telephone intervievHng system at the Survey Research Institute (SRI) at 
Comeh University.4 


4 "The Survey Research Institute (SRI) is a full-service survey research facility at Cornell University. 
The primaiy mission of SRI it to conduct surveys and provide survey research services to Cornell 
University faculty, students, and administration, federal, state, and local government agencies, other 
nonprofit organizations, and other organizations in need of survey research work. SRI is committed 
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In order to measure common instructional strategies within AIS classrooms, we 
reviewed and selected appropriate items from the National Education Longitudinal Study of 
1988 (NE LS: 88-94). Previously tested for rehabUity and vahdity by the US D epartment of 
Education's National Center for Educational Statistics, the items included measures of general 
pedagogy, classroom organization, and subject- specific items related to the teaching of Enghsh 
or mathematics. While many other measures of instruction could have been used, the NE LS 
measures serve our purpose of rehably, measuring whether basic instructional strategies differ 
between AIS and non-AIS teachers. 

Sampling 

The principal goal of our samphng strategy was to balance a representative selection of 
districts (as well as the population of students the districts represent) against a need to 
represent wealthy and poor, urban, suburban, and rural communities. This goal encouraged us 
to utilize both cluster and stratified sampling approaches. 

In designing our data collection, we used a cluster samphng approach, selecting 
districts, then selecting schools within those districts, and finally selecting teachers within the 
schools (See T able 1). Overall, we selected 121 districts and then 246 schools (including 166 
high schools and 80 middle schools) in these districts. We then surveyed the principal in each 
school. T 0 provide an in-depth documentation of AIS in classrooms, we then chose a subset 
of these districts (70) to survey more than 500 teachers in grades 7-12, Eor the purpose of this 
study, we considered an 'in-depth' district to be a district where the middle and high school 
principals (if both exist) and four teachers completed surveys (two Enghsh and two math). 

This requirement did not hold for districts with fewer than four English and mathematics 
teachers, in which case we would select all the Enghsh and mathematics teachers. These 246 
administrators and 500 teachers give us an overall picture of AIS. 

District Level 

We selected the NY C Public Schools, the Big Eour large urban districts (i.e., Buffalo, 
Rochester, Syracuse, and Y onkers), and then divided the rest of the upstate districts into low- 
wealth, mid- wealth, and high- wealth districts. The wealth categories were created using the 
Combined Wealth Ratio (CWR; a composite index of property and income wealth available to 
each school district). Erom the 638 non-Big Eive school districts with high schools, we selected 
a total of 116 districts, 39 low- wealth districts, 38 mid- wealth districts, and 39 high- wealth 
districts. Letters were mailed to superintendents of selected districts in March 2003, asking 
them to respond if they did not want their district included in the study. 

School Level 

We selected both high school and middle school principals to interview. In NY C, we 
randomly selected 45 high schools and 20 middle schools to participate in the study. Eorty-five 
(45) NY C High School Principals participated and 20 NY C Middle School Principals 
participated. 


to offering state-of-the-art technology to its clientele, striving for the highest possible quality in 
performance while maintaining the highest possible ethical standards of conduct." 

(http:/ / www.sri.comell.edu/ about.html) 
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With regard to the Big Four, we randomly selected four high schools and two middle 
schools from Buffalo, and two high schools and two middle schools each from Rochester, 
Syracuse, and Y onkers. Following the principal surveys, one of the Big Four districts declined 
to participate.^ Thus we dropped two principal interviews already collected from this district 
and did not collect any teacher data from this district. 

From the remaining school districts with high schools in the state, we selected one high 
school from each of the 116 selected districts. If there was more than one high school in the 
district, the high school was randomly selected (fewer than 10% of these districts have more 
than one high school). Next, we randomly chose 17 middle schools from the 39 low- wealth 
districts, 17 from the 38 mid- wealth districts and 17 from the 39 high- wealth districts. If there 
was more than one middle school in the district selected, the middle school was randomly 
chosen. Some districts had combined high school and middle schools and hence there was no 
middle school principal. 

Teacher Level 

0 ur initial sampling approach assumed that the 2001-2002 NY S Personnel Master 
File^ would aid in the identification and description of AIS teachers within NY S schools, but 
we found dramatic underreporting of AIS course instruction by T eacher Assignment Code. 
This process led to a redesign of the teacher sampling strategy. From the schools that were 
selected to participate in the study and whose principal was interviewed, we selected five 
Enghsh and five mathematics teachers teaching at least one section of the subject area in grades 
7-12 from the BEDS Personnel Master File database. 

• In NY C, we randomly selected five middle and five high schools from the set of schools 
where we had successfully surveyed principals. We then randomly selected five math 
teachers and five English teachers from each school resulting in a target number of 100 
teachers from NYC. We asked all teachers specific classroom information (i.e., number of 
students, number of AIS students, whether the teacher is responsible for providing AIS 
services in the class). 

• Forthe Big Four districts, we selected 10 Enghsh and 10 math teachers (grades 7-10) from 
the high schools and middle schools in three of the four Big Four districts. 

• From the 116 upstate districts, we chose 18 districts from each of the stratum (i.e., 18 of 39 
low-wealth districts, 18 of 38 mid-wealth districts, and 18 of 39 high-wealth districts). 

0 nee the 54 districts were chosen, we randomly choose 10 teachers, five math and five 
English teachers, in grades 7-12 from each of the districts. In schools with fewer than six 
Enghsh or mathematics teachers, we did not sample but rather attempted to survey the 
universe of teachers in the building. 


5 Per our human subjects agreement with each district, the name of the non-participating district will 
remain confidential. 

6 The Personnel Master File is part of the Basic Educational D ata System, update annually with 
survey data collected eveiy 0 ctober from eveiy teacher and administrator in the state. 
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Table 1 

Summaiy of Sampling Strategy for D ata C oUection 



# Districts 

# Schools/ 
Principals 

Teachers 

(Schools) 

NYC 

1 

65 

51(1) 

Big Four 

3 

16 

38(3) 

Non-Big Five 

136 

172 

422(56) 

Lowest 1/ 3 CWR 

43 

56 

155 

Middle 1/ 3 CWR 

47 

59 

142 

Highest 1/ 3 CWR 

46 

57 

125 

Total 

140 

253 

511(60) 


Weighting 

D epending on the policy question posed, it may be more valuable to understand the 
proportion of districts engaging in a particular practice, or possibly the proportion of students 
that are impacted by a given practice. As such, we apphed two different weighting schemes to 
our sampled data in order to generahze the findings across the state. Specifically, the samphng 
design affords us the opportunity to properly weight the survey responses for the population 
of districts within each stratum, and separately for the population of students in each stratum. 
We calculate these weights to ensure that our sample of principals and teachers accurately 
represent the population of districts in each wealth stratum, and separately the number of 
students being educated in each of the three strata. T o generate these weights, we calculated six 
separate weights - two for principals only, two for teachers only, and two for the combined 
sample of principals and teachers. One pair of weights is calculated to allow for generalization 
of the findings to the population of districts, and the second is for generalization of findings 
to the student population in each stratum across the state. 

We calculated a district weight by dividing the total number of districts in each of the 
three wealth strata by the number of districts in our sample. Since we have multiple 
respondents in many districts, we divided the district weight by the number of principals, 
teachers, or both combined, and assigned each respondent the resultant weight. For example, if 
120 districts are in the upper stratum and we have data from 40 districts in the same stratum, 
the district weight is 3 (120/ 40). If two principals and 10 teachers are in the sample from a 
given district, the individual principal weight is 3/ 2 or 1.5. The individual teacher weight is 
3/ 10 or .3 and the combined weight for analyses using both principal and teacher data is 3/ 12 
or .25. For the Big Four districts, we only have data from three of the four districts and hence 
each of the district weights is 4/ 3 or 1.333. 

The student enrollment weight is calculated much the same way, but by using the 
aggregate enrollment of the districts in the strata and the total enrollment of each district. For 
example, if the aggregate enrollment of districts in the poorest third of the non-Big Five 
districts is 200,000 and a given district has a total enrolment of 10,000 students, the district 
weight is (200,000/ 10,000) or 20. Subsequently, if we have 15 respondents (2 principals and 
13 teachers) the weight for each respondent is 20/ 15 or 1.333. 
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Analytic Methods 

Univariate Approaches 

We begin by presenting simple univariate means and standard deviations for two 
distinct sets of verifies. The first set of variables is related to the following AIS issues and 
rehes on responses from principals only: 

- Selection and termination criteria for students in AIS programming 

- Persons involved in the decision-making to assign students to AIS 

- Student participation 

- Planning 

- Scheduhng 

- Staffing 

The second set relies on teacher response and is related to the following issues: 

- Instructional and classroom organization practices 

- Instructional planning 

- Classroom environment 

We then test for significant statistical differences between subgroups of respondents. Using 
the principal data to examine the logistics and scheduhng of AIS, we analyze whether bi- variate 
differences exist between principals in NY C, the Big Four large city districts, and the non-Big 
Five districts. Subsequently, we use teacher data to assess any differences in instructional and 
classroom organization strategies between teachers responsible for AIS instruction and those 
teachers who are not responshale for the provisions of AIS. 

0 ur survey also ahowed for the cohection of data from open-ended questions from both 
teachers and principals. We weave some of this qualitative data into our findings to help clarify 
and exphcate the findings. Future studies will more fully describe this data. 

Finahy, we hnk the survey data with four years (1999-2003) of district performance and 
demographic data to examine the multivariate relationships between community wealth, 
district fiscal, demographic, and performance measures, and the AIS organizational and 
instructional strategies reported by principals and teachers. We use the publicly available 
Chapter 655 and School Report Card datafiles (1999-2003) to investigate demographic and 
performance differences and similarities between schools with different models of AIS 
programming, staffing, and scheduhng.’ 

Multivariate Approaches 

In order to effectively describe AIS practices and test for substantive differences across 
districts, we selected methods that would inform discussions of key AIS practices and the 
likelihood of their implementation given various contexts. The nested nature of the data 
(multiple teachers within districts) requires techniques other than ordinary least squares (0 LS) 
regression models. Most of our regression models use either binary or categorical dependent 
variables as we tried to predict differences across contexts with specific AIS strategies. These 
robust regression estimates adjust the standard errors found in the correlated residuals 


’ See http:/ / www.emsc.nysed.gov/ irts/ 655report/ home.html and 

http:/ / www.emsc.nysed.gov/ irts/ reportcard/ home.html for the publicly available data. 
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stemming from the nested structure of the data (e.g., individuals within districts).^ D oing so 
properly accounts for the district- effect on the participants' responses. 

T 0 serve as indicators of school district performance, we selected 8*^ grade ELA scores 
(district aggregates) both at the absolute level in 1999, and then a measure of change over time 
between 1999 and 2003. To calculate the change variable, we regressed the mean of the 2001/ 2 
and 2002/ 3 scores on the mean of the 1999/ 0 and 2000/ 1 scores and saved the residuals as a 
new standardized variable. Hence, the indicator for change in 8*^ ELA performance over time 
has a mean of zero and one standard deviation above the mean of one. We chose these 
indicators given the time period during which we are collecting principal and teacher data on 
AIS programming (2003). Using the 1999 achievement levels, and then the growth in 
achievement in the initial years of AIS programming, we are able to analyze whether prior 
performance levels or gains in performance over time predict current AIS programming. It is 
not prudent at this time (though possible in future years) to use AIS programming to predict 
current achievement levels given the lag time necessary for the treatment (i.e., AIS) to have an 
effect on achievement. 

The estimates reported are odds- ratios and are easily interpreted. Eor example, a value 
of one (1) indicates even odds of occurrence at different levels of the independent variable or 
between comparison groups. Any significant value greater than one indicates an increased 
hkehhood of occurrence (e.g., a value of two indicates the practice is twice as likely as the 
comparison group) and any significant value less than one indicates a reduced likehhood of 
occurrence when compared with the comparison group (e.g., an odds ratios of .50 would 
indicate that the practice is only 50% as likely to be used as the comparison group). 


Findings 

We report our findings in four steps. We begin with simple descriptive (univariate and 
bi- variate) measures of organizational/ structural strategies followed by instructional strategies 
and priorities. We then report multi-variate regression results for the organizational/ structural 
strategies and then for instructional strategies. 


AIS Organizational/ Structuial Arrangements 

Student Assignment 

How are students enrolled and dismissed from AIS programs? In this line of 
guestioning, principals were asked to weigh various criteria used to warrant the provision of 
AIS services to students as well as indicate the people typically involved in this decision. In 
summary, administrative decisions rely heavily on standardized tests, report cards, and 
guidance counselor recommendations to both provide and suspend AIS services for students. 
Criteria such as classroom behavior and student attendance do not factor heavily in this 
process. 


8 We used xtlogit and ologit in Stata to conduct logistic regression analyses while adjusting for the 
effect of clustered responses. 
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G enerally across the state, formal report card grades and student performance on 
standardized tests factor heavily into the decision of administrators to provide AIS services to 
students. Of the two, student performance on standardized exams (including state exams) is 
more important in AIS placements. Principals representing 70% of aU districts indicate that test 
scores on these exams are very important. Student attendance and classroom behavioral issues 
seem fairly unimportant in the decision-making process. 0 f those adults able to make 
recommendations regarding AIS placement, it appears that the recommendation of the 
guidance counselor is most important, with teacher and then parent recommendations being 
less important. 

Y et when principals were asked how they would change their current AIS program, 
several noted that they would make improvements to the process of identifying AIS students. 
For example, principals commented that they would identify students earlier. One principal 
commented, "What we need to do is more timely diagnostics so we catch problems earlier." 
Another principal explained the problem of identifying students for AIS services on outdated 
4*^ grade test scores, "There are no exams in 5th and 6th grade, just the 4th grade exams. They 
come into 7th grade based on their 4th grade exams." Still another principal commented that 
receiving the scores from NY S earlier would help, "I would like to see the test results back 
from NY S earlier, to identify earlier." 

Other principals commented on the characteristics of the student population identified 
for AIS. 0 ne principal noted: 

I would take the special education students out of AIS - 1 don't believe they need that and it 
takes away from time that could be given to other students. They're already identified and 
receiving special education services and they're supposed to receive additional services, 
which is kind of a double dip but we're mandated to do that. 

0 ther principals noted that they would expand the program to serve more kids in need of 
assistance. For example, one principal commented, "I would have more funding for additional 
staff so that we could service the kids who got twos on the state exams as well as we service the 
[students] who got ones." 

Interestingly, principals in NY C schools reported a heavier reliance on a wider number 
of criteria used to identify students for AIS in comparison with their counterparts upstate. 
While aU districts tended to rely on performance exams equally, administrators in NY C 
indicated that report cards were more heavily utilized in their schools compared to upstate 
districts. Similarly, NY C districts relied more on parent recommendations, classroom behavior, 
and attendance than upstate schools. 

Are the same criteria and individuals involved in the decision to terminate services? 
Report cards, tests, and the recommendation of the guidance counselor are most sahent in 
terminating AIS services for students. Parents, attendance, and classroom behavior are not very 
important. Y et, when principals were asked about what they would change with their current 
AIS program, some noted the need for more flexibility in testing to decide whether to 
terminate. For example, one principal commented, "I would give the student the ability to test 
out of AIS. At this point, the only way a student can get out is with the next State exam." 

Principals were also asked to indicate which people were more or less involved in AIS 
programming decisions. Individuals were ranked on a scale as either not involved or very 
involved. Predictably, administrators and school counselors are very involved, as are teachers. 
Statewide, principals representing 31.1% of districts said that parents were involved in the 
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decision to grant AIS services. Twenty-eight percent (28%) of districts report that parents were 
important in the termination of AIS services. What stands out, however, is the absence of 
student involvement in the AIS programming decisions. Principals representing 63.7% of 
districts stated that students were not very involved in these decisions. 

In contrast, NY C districts tended to have much greater involvement with parents and 
students in AIS programming than did the non-NY C districts. For example, the average 
involvement score for student participation in the AIS programming process was a 2.08 among 
upstate districts. However, among NY C schools, that same score was 3.32. When asked to 
name other individuals typically involved in this process, a great number of principals stated 
that central office administrators like directors of curriculum, as weU as school psychologists 
and social workers, were also veiy involved. 

Enrollment 

In general, AIS services are provided to far greater proportions of students in urban 
districts than among upstate districts. Statewide, principals report that approximately one- third 
(31.1%) of their students receive AIS services. Among these students, 36% also receive special 
education services. By comparison, principals report that 56% of their students in NY C 
schools and 51% of their students in Big Four schools receive AIS programming. More than 
half of NYC AIS students also qualify for special education services as weU. 

However, principals in non-Big Five districts report that 6% of students that quahfy 
for AIS do not receive services for one reason or another. In NY C, principals report that 
15.3% of their students qualified to receive AIS do not actually receive those services. When 
asked to identify the primaiy reason for the discrepancy between the number of students who 
qualify for AIS but do not receive AIS instruction, the answers fell into one of three categories. 
First, principals stated that student absenteeism from school and AIS classes is a large issue. 
Some students simply avoid the AIS class and/ or school altogether. Second, as there are no 
rules about compulsory AIS attendance, parents will often disallow students to attend AIS 
classes. Third, some schools have tremendous difficulty scheduling all students for AIS given 
their tight course schedules. 0 nly a handful of piincipails mention^ transportation issues. 

Staffing 

In general, multiple educators are employed to cany out AIS services. In 55% of 
districts, principals reported that designated AIS teachers staff AIS services. Over 60% of 
districts have special education teachers employed in AIS services. 

We inquired as to collaborative arrangements for AIS instruction and teaching models. 
Specifically, principals were asked whether co -teaching or consulting models existed. For 
example, this would be when special education or AIS teachers pushed into programs within 
regular classes. Sixty- two percent (62%) of districts followed this approachThis lends credence 
to the earher findings that found about the same percentages for push-in and pull-out AIS 
service dehveiy mechanisms. 

Y et, many principals spoke about the need for additional staff, most specifically to 
reduce class size and provide more individualized instruction. One principal commented, 
"Increase staffing and increase time for planning of dehveiy of individuahzed services." In fact, 
when asked what would they change about their current AIS program, more principals 
responded that they would increase the number of staff than any other reported change. Staff 
included teachers, counselors, social workers, and paraprofessionals. One principal explained. 
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"I would try and hire more staff or faculty so that we could differentiate some instruction in 
the areas that we currently do not in science and in Enghsh, and also we'd hke to add social 
studies." 


Scheduling 

AIS program patterns are multifaceted, and as such are difficult to generahze across the 
state. Following a brief introduction to the section, we present and discuss some basic patterns 
to AIS program scheduhng as weU as some relationships between scheduhng models. Though 
multiple scheduhng models exist and in any one school several models may be in place, we 
suspect that the prevalence of schools with two or more AIS dehveiy models per subject is 
actuahy small. The correlation evidence presented below supports this assertion. 

Overah, Enghsh and math AIS programs are more common than other subject area 
AIS programs. Principals report that AIS is universally provided for English and Math 
subjects. While a httle over eighty percent (80%) of districts report some social studies AIS and 
76% report some science AIS programming. 

For each subject area, the provision of AIS services occurs both during and outside of 
regular school hours. In the case of English AIS for example, approximately half of all districts 
manage AIS Enghsh either before or after school hours. However, we are unable to tell what 
programs are exclusively "in-school" versus "out-of-school", or a blend of the two. As such, 
these overlapping conditions frustrate a clear and cogent description of AIS program 
scheduhng patterns or model identification. 

Interestingly, though more than half of ah districts run AIS before or after school, 
when principals were asked what they would change about their AIS program, many 
commented that they would prefer to integrate the programs into the school day. For example, 
one principal said, "I would find a way to include more AIS during the school day. What 
happens is the kids could go to any school in the district and because of that, it is hard to 
schedule AIS after school. So I would hke to buhd AIS into the schedule. I would need a httle 
more money." Principals' comments suggest that when AIS is before or after school, students 
do not necessarily attend. Y et, if AIS is going to be effective, principals argue, students should 
be reguired to go. This is better ensured if AIS is provided during tiie school day. Another 
principal commented, "Provide funding to do AIS as part of the regular program during the 
day instead of after school. . . add an extra period so the students don't see as it as optional." 

A chief difference among AIS program scheduhng appears to be the strength of their 
association with regular subject area classes. Akin to special education dehveiy models, the 
scheduhng of AIS programs may be inclusionaiy in nature or held in self-contained 
classrooms. It is also true that schools may elect to schedule AIS programs in more than one 
way. We beheve there are four general categories to AIS scheduhng: 

Model 1. AIS inside the regular classroom 
(Characterized as inclusive) 

Model 2. AIS held during classtime, outside the regular classroom 
(Characterized as self-contained, pull-out) 

Model 3. AIS held in addition to the regular classroom, in lieu of electives 
(Characterized as self-contained and additive) 

Model 4. AIS held in place of the regular class 

(Characterized as self-contained and supplanting) 
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In the first model, principals report that more than a third of students receive English 
AIS programming within the regular education classroom. We suspect that this first model 
includes AIS activity and programming during the regularly scheduled class session. However, 
it is unclear if the schedule involves instruction with a designated AIS teacher, or the regular 
education teacher performing AIS activities. 

In the second model, principals report that a third of all students are drawn outside of 
their regular English class for AIS programming. Although students are drawn out of and thus 
away from the regular classroom, we believe principals still interpret these programs as 
supplementaiy or additive because they are so closely associated with the regular classroom. 
There is no statistically significant difference (p<.01) between these proportions among the city 
and upstate school districts. However, the means trend towards a higher proportion of 
students (47%) in NY C participating in this pullout model, compared with students (32%) in 
upstate school districts. 

The third model appears to have less connectivity with regular education classrooms. 
When asked if students were placed in a specific AIS class that meets in addition to regular 
English class, principals representing 82% of districts respond affirmatively. Note however that 
this proportion is somewhat higher and somewhat in conflict with the second model described 
above. This third AIS model appears to be administered separately from the regular subject 
area class, and is attended by students in lieu of their electives and traditional study halls. In the 
case of English AIS, principals representing 48% of districts (and 47.7% of students) report 
that AIS is offered in lieu of study halls, and 44% of districts allow for electives such as art, 
music, and foreign language to be replaced. There is no statistically significant difference 
between these proportions among the city and upstate school districts. There is no statistical 
difference between the proportion of districts affected in this "additional" model among the 
city and upstate school districts. 

In the last model, and consistent with prior research, we asked the guestion of whether 
academic intervention services were being offered in place of regular education or in addition 
to regular education classes. Prior case study research couched these differences as either 
supplementary (an additive program service) or supplanting (a replacement program service) 
(see Sipple, Killeen, and Monk, 2004). As was the case in this prior research, a number of 
districts representing a large proportion of children do report that AIS programs are being 
substituted for regular education classes. Eor example, principals representing 13% of districts 
offer English AIS instead of regular classes. This finding is also reinforced by answers to the 
contrary guestion. Principals representing 81% of districts report that English AIS programs 
are offered in addition to regular classes. These basic patterns hold across subject areas and do 
not appear to differ between urban and upstate districts. 

As previously mentioned, we believe some schools elect to administer AIS programs 
through a variety of scheduling models. G iven this, we also calculated the strength of the 
association between principal responses to survey items using Pearson correlations on 
unweighted response data^. In the case of English AIS, if programs are run either before or 
after school, then there is a tendency to not run the programs in place of electives (r = - .14; p < 
.05). This type of modest association is consistent for subject areas like English and 


9 The strength of several correlations and their significance levels are reported as necessary. Tables 
showing the correlations for each survey item in this section are available upon reguest to the 
authors, but are not included in this report. 
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mathematics. In the sciences, however, the strength and frequency of the item associations 
increase, again likely due to the inability to fit additional sections of AIS into the regular school 
day. 

AIS science scheduling may create unique issues within schools given the need for 
specialized lab equipment that is not easily moved around in school facilities. Again, we draw 
on relationships between responses from principals. If AIS science is offered in place of the 
regular education class, it appears to be associated with replacing electives as well (r = .28; p < 
.05). The strength of this association indicates that at least some of the "supplanting" AIS 
models mentioned earlier include AIS science classes. Interestingly, if AIS science is offered in 
addition to the regular education class, then it is slightly more associated with the replacement 
of electives (r = .32; p < .05). In this instance, it seems likely that AIS science may actually be 
offered as a separate lab. The association between AIS science being offered outside of the 
regular class and in a designated AIS academic lab is modestly strong (r = .33; p < .05). 

In summary, AIS programming is delivered for each subject area (English, math, 
science, and social studies) in the vast majority of NY S school districts. A key distinction 
among AIS scheduling models is the degree to which the AIS model is associated with the 
regular education class, similar to the typical variations in self-contained and inclusive special 
education service models. Though schools may deliver multiple AIS models for each subject, 
the prevalence of such practices among districts appears small. 

Instruction 

0 ur focus on AIS classroom instruction hinges on whether instructional strategies in 
AIS classes differ from those found in regular subject area classes. The survey asked teachers to 
focus on one particular class they taught during the week. We then asked teachers to describe 
this class, and to respond to a set of general instructional questions and subject- specific 
instructional questions related to this class. 

Whereas the principals report a global estimate for AIS participation, we asked teachers 
to report on a specific class period. Over the full sample of more than 500 teachers, we have in 
essence, a random sample of classes taught across the state at a given point in time. 

Responding to a specific class they are actually teaching, we asked teachers to report both their 
class enrollment and AIS enrollment, and whether or not the teacher is required to provide 
AIS instruction in that class. For the Big Four and other non-NY C Needs Resource Capacity 
Categories, the teachers' reports do closely mirror the principals. For instance, teachers in Big 
Four districts report 44% of their students as AIS students (43% for principals) and in average 
need districts, teachers report 32% AIS students and principals 28%. In NY C, however, the 
teachers (29%) report a far smaller proportion of AIS students than do principals (56%). 

The typicd classroom portrait based on the current teacher sample has, on average, 19 
students, with eight of the students receiving AIS services. As a proportion (43%) this figure is 
slightly higher than the figure reported by their principals. T eachers in one- third (33% ) of 
districts had no AIS students in their classroom, teachers in 14% of districts reported having 
only AIS students in their classrooms, and teachers in 50% of districts reported a mixed 
classroom with both AIS and non-AIS students. In addition, 17%, or approximately three of 
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Table 2 

AIS Class Descriptive Statistics (T eacher Report) 


N eed Resource T eacher Required to 

# of Students in 

# AIS Students in 

% AIS Students 

Capacity Index 

Give AIS? 

Class 

class 


Y es Mean 

21.7 

15.9 

76.8% 


Std. D ev. 

7.9 

10.6 


NYC 

No Mean 

27.3 

3.5 

13.8% 


Std. D ev. 

6.2 

6.0 



Total Mean 

25.9 

6.3 

28.7% 


Std. D ev. 

7.0 

8.9 



Y es Mean 

24.0 

15.4 

65.6% 


Std. D ev. 

9.3 

12.1 


Big Four 

No Mean 
Std. D ev. 

25.6 

12.2 

5.6 

16.4 

18.9% 


Total Mean 

24.8 

10.8 

43.7% 


Std. D ev. 

8.5 

11.8 



Y es Mean 

14.7 

9.2 

77.2% 


Std. D ev. 

8.0 

6.1 


High Need Urb/ Sub 

No Mean 

21.2 

4.4 

21.0% 

Std. D ev. 

4.6 

6.3 



Total Mean 

18.9 

6.0 

40.2% 


Std. D ev. 

6.6 

6.6 



Y es Mean 

11.6 

7.7 

80.4% 


Std. D ev. 

7.4 

4.5 


High Need Rural 

No Mean 

18.0 

3.0 

16.6% 

Std. D ev. 

5.1 

3.9 



Total Mean 

16.4 

4.2 

31.9% 


Std. D ev. 

6.3 

4.5 



Y es Mean 

16.3 

8.7 

69.4% 


Std. D ev. 

9.4 

5.9 


Ave N eed 

No Mean 

21.6 

2.7 

13.3% 


Std. D ev. 

5.1 

3.0 



Total Mean 

19.8 

4.8 

33.0% 


Std. D ev. 

7.3 

5.1 



Y es Mean 

17.1 

9.8 

66.3% 


Std. D ev. 

9.1 

8.9 


Low Need 

No Mean 

22.6 

1.5 

7.6% 

Std. D ev. 

5.4 

3.5 



Total Mean 

21.3 

3.5 

21.4% 


Std. D ev. 

6.7 

6.2 



Y es Mean 

16.0 

9.7 

73.5% 


Std. D ev. 

8.9 

7.1 


Total 

No Mean 

21.6 

3.0 

14.4% 


Std. D ev. 

6.1 

4.4 



Total Mean 

20.0 

4.9 

31.5% 


Std. D ev. 

7.5 

6.1 
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the 19 students, were receiving non-academic AIS services such as counseling, nutrition, or 
health assistance. T eachers also reported, on average, that an additional 11% of students in this 
classroom could benefit from AIS services but were not receiving them. This latter figure is 
within the range reported by principals. 

Comparing classes taught by a teacher responsible for providing AIS instruction versus 
those who are not responsible, the class makeup is guite different (see T able 2). Class sizes are 
consistently smaller for teachers responsible for AIS instruction (e.g., 22 vs. 27 in NY C, 15 vs. 
21 in High Need Rural D istricts) and the proportion of AIS students is consistently higher 
(e.g., 77% vs. 13% in NYC, 66% vs. 19% in the Big Four, and 80% vs. 17% in High-Need- 
Rural). 

Principals in non-Big Five districts report that 6% of students that gualify for AIS do 
not receive services for one reason or another. In NY C, however, principals report that 15.3% 
of their students gualified to receive AIS do not actually receive those services. 

Modal Model of Instruction 

T eachers report that whole class instruction is the most common mode of instructional 
delivery. By way of actual pedagogy, popular deUveiy methods include straight lecture, whole 
group discussion, and/ or oral responses to individual student guestions (See T able 3). 

T eachers were asked the percentage of time they spent on various practices necessary to 
provide instruction in their classroom. On average, teachers representing 52% of students 
report spending most of their time (50- 1 00% ) instructing the whole class. In this sense, 
teachers spend the majority of student seat time focused on instructional delivery. They report 
only spending small amounts of time spent on maintaining order and discipline and 
performing routine administrative tasks. 

T eachers were also asked more specific guestions about how often they use more 
detailed teaching methods and instructional media (see T able 4). Most students, according to 
teachers, are engaged in oral guestion and answer sessions several times per week. T eachers 
often lead whole group discussions, and other times students work cooperatively in teams or 
complete individualized writing assignments in class. Several teaching methods are less 
freguently used, including lecturing as weU as computer work. 

AIS versus non-AIS instruction 

T 0 examine whether AIS instruction is distinct from non-AIS instruction, we 
compared the responses of teachers who were responsible for AIS instruction in this particular 
class to those teachers who were not responsible for AIS instruction in the class. AIS teachers 
distinguish themselves from non-AIS teachers through their utilization of small group 
interactions with students (see T able 3). While lecture and whole class discussion was the 
modal model for all teachers, those teachers responsible for delivering AIS instruction reported 
using small-group and individual instruction more freguently, and using lecture/ whole group 
methods less freguently than those teachers not responsible for delivering AIS in this 
classroom. They report more individualized or one-on-one attention with students. Although 
teachers report administering tests and guizzes less freguently in AIS classes than in non-AIS 
classes, they are more likely to teach test taking skills (see T able 5). 
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T able 4. Frequency of Classroom Instructional Methods by T eacher Type (by AIS T eacher/ N on-AIS T eacher) 



Mean for 
Teachers NOT 
Responsible for 
AIS instruction 

2.64 

0.90 

1.78 

2.60 

3.65 

1.35 

2.32 

2.33 

0.66 

Mean for Teachers 
Responsible for AIS 
instruction 

O 

C^* 

* 

* 

CO 

T— 1 
T— 1 

* 

* 

1.74 

2.60 

3.59 

LO 

CO 

T— 1 

* 

* 

2.44 

CNl 

CO 

CNJ* 

* 

* 

0.62 

CD 

C/5 

n 

O 

Oh 

C/5 

<D 

pc; 

everyday 

T— 1 
T— 1 

CT5 

CNl 

05 

T— 1 

30.8 

71.4 
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05 

T— 1 

25.9 

25.3 

2.0 

Almost 

eveiyda 

y 

24.7 

cr> 

cn 

c^‘ 
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26.7 
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21.8 
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24.5 

29.3 

32.2 

CNl 
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T— 1 

1- 

2x/mo 

nth 

LO 

LO 

19.3 

28.8 

8.2 


O 

LO 

T— 1 

O 

T— 1 

6.2 

27.2 

Never 

/rarel 

y 

24.7 

42.1 

24.0 

cn 

o 

T— 1 

0.7 

33.3 

T— 1 

CO 

6.2 

58.5 

Mean 
for ALL 
teachers 

2.42 

05 

05 

1.65 

2.59 

T— 1 

CO 

cn 

1.49 

2.45 

2.36 

.75 

How often do you use 
the following teaching 
method or media 


Lecture 

Use Computers 

Use Audio-visual 
material 

Have teacher- led whole 
group discussion 

Have students respond 
orally to questions on a 
subject matter 

Have student- led whole 
group discussions 

Have students work 
together in cooperative 
groups 

Have students complete 
individual written 
assignments or 
worksheets in class 

Have students give oral 
reports 


p<.10, **p<.05 
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Several teachers (non-AIS and AIS), when asked what they would change about their 
AIS program, noted that they would decrease the number of AIS students in a class or group. 
In fact, this was one of the most common responses. T eachers made such comments as, "I'd 
like to get class size down," "More teachers with support of aides. Lower teacher to student 
ratio with administrative support," and "I would have smaller groups, no more than 10, for 
those that gualify. 0 n an as need basis if they need it, for a shortened time period." 0 n the 
whole, teachers reported spending comparable amounts of time on routine administrative tasks 
and on issues regarding classroom behavior. We found no evidence to support a claim that 
AIS teachers contend with greater classroom confhct and student disciphne issues than non- 
AIS teachers. 

Classroom instruction strategies do differ across academic subject areas between the 
two groups of teachers. However, these differences are not dramatic and only appear in certain 
instances. For example, AIS Enghsh teachers reported reguiring students to read novels, plays, 
essays, etc. less often than non-AIS E nghsh teachers (see T able 6). Y et, there appear to be no 
differences in letting children choose their own reading material, discussing assigned reading 
materials, or even the focus on technical aspects of writing. In the case of math, teachers in AIS 
classrooms report more freguent attention to the importance of math in daily life. However, on 
many other common items, there appear to be no other significant differences. Issues such as 
the memorization of facts, rules and steps, understanding the nature of proofs, and even items 
such as performing calculations with speed and accuracy appear to receive even attention in 
both the AIS and non-AIS classroom. 

Interestingly, when math teachers were asked whether they focus on increasing 
students' interest in math, there was no difference among AIS vs. non-AIS classrooms (see 
T able 7). This is significant given that one of the major barriers to the effectiveness of AIS 
reported by teachers was student motivation and participation. In fact, student 
participation/ engagement was the most common response by teachers when asked what the 
biggest challenge for AIS students in meeting the learning and graduation reguirements. For 
example, one teacher noted, "Motivation; lower level learners struggle so much it's hard to get 
them interested." This included getting students to AIS as well as their motivation once they 
were in the AIS setting. For example, one teacher commented, "Making sure the students who 
need the services are in the class." Another teacher noted, "AU of the kids who did their 
homework regularly are out of AIS this year. Those who didn't are stiU in it." 

Multivariate Analyses of AIS Programming 

Having presented basic descriptive statistics, freguencies, and correlations of AIS 
organization and practice, we relate these practices with wealth, performance, and relevant 
teacher and classroom characteristics. In other words, we ask whether certain practices are 
more or less prevalent in wealthy communities, in districts with higher levels of academic 
performance, or in districts that have made greater gains in performance, since the 
implementation of the new Regents Standards and AIS regulation. In this study, we focus on 
three sets of AIS- related practices: scheduhng, elements of instruction, and teacher planning 
for AIS. 
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Table 7 

D escriptive Statistics for Mathematics T eacher Instruction 
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Scheduling 

We first examine the contextual differences associated with (though not necessarily 
caused by) the scheduling of AIS instructional time for students. Table 8 represents Enghsh 
AIS scheduhng and T able 9 represents the scheduhng of AIS in mathematics. In examining the 
possible times to schedule student AIS, three guestions are important to answer: 1) Is AIS 
scheduled before and/ or after regular school hours? 2) Are students missing electives or using 
study haUs to 'fit in' AIS class time? 3) Is AIS class time for students scheduled during the 
regular Academic (Enghsh or Mathematics) class, outside of the regular Academic class, or in 
addition to the regular Academic class? 

We have substantial gualitative evidence from principals and teachers that a major 
obstacle of their AIS programming is simply getting students to show up. Many noted that 
when AIS is not reguired (e.g., before or after school hours or voluntary in-school drop- 
centers) many students do not attend the designated AIS sessions. 

The regression findings suggestthat while the absolute level of performance prior to 
implementation of AIS is unrelated to the availability of before or after school scheduhng 
options, district change in performance over time (8*^ grade E LA scores) is positively related to 
scheduhng AIS before or after school. It is important to note here that these scheduling 
options are not mutually exclusive and many districts use any number of combinations of 
scheduhng options. N either community wedth (property wealth or income wealth) nor the 
proportions of poor children (% frpl) are related to the use of before or after school AIS 
instruction. D istrict size is positively associated with before and after school AIS with larger 
districts more likely to include the strategy. In terms of where the districts are located, NY C, 
the Big Eour, and small city schools ah have odds ratios over two (2) for Enghsh, though only 
small city schools are statisticahy significant different than the comparison group of rural 
districts. There are no significant differences between suburban and rural district practice, 
indicating the prevalence of this practice is roughly consistent across these districts, once 
controhing for size, wealth, and performance. 



Table 8 - Logistic Regression Estimates of English AIS Scheduling Options (Values are Odds Ratios) 
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A popular concern is that some students wUl be unable to take elective courses as their 
AIS classes are scheduled in place of the electives. Is this scheduhng phenomenon more likely 
to take place in poor districts, with poor students, or in small rural districts with fewer 
scheduling options? The findings suggest that the scheduhng of AIS in place of electives is not 
associated with the measures of performance, wealth, or size, and is only related to location in 
that the Big Four educators are less likely to report this practice than those in rural areas. While 
only approaching significance, it does appear that this practice may be more likely in wealthier 
and more suburban communities than in the urban or rural areas. Further investigation should 
attempt to determine if this is because of the reduced rehance of before and after school 
options in the suburban districts, and hence greater pressure placed on student schedules, than 
in districts where before and after school times are used for AIS. 

0 nee controlhng for uibanicity, greater proportions of poor children are strongly 
related to using study halls for Enghsh AIS services. The use of study halls for AIS is also 
positively related to district enrollment. Finally, districts averaging one standard deviation more 
than others on the 1999 8*^ grade E LA exams are one and one-hdf times as likely to use study 
halls. Perhaps this signals a complacency of higher performing districts to simply use study 
haUs rather than creating more specific AIS programming. 

The AIS regulations reguire additional instruction for under-performing students, 
though the details of exactly how to implement are left up to local districts. We have already 
shared evidence that most school districts are scheduling AIS time in addition to regular 
academic classes, though we also see that some districts are substituting an AIS version of the 
course for the regular academic class. Alternatively, districts may provide additional instruction 
via a class that pulls students out of their regular subject area classes. Another model is to push 
AIS instruction into the regular English class, or reguire the regular Enghsh teacher to provide 
the AIS services. Again, many of these options are not mutuahy exclusive. 

The regression results indicate that higher performing districts are more likely to offer 
AIS instruction during the regularly scheduled Enghsh classes (possibly via pushing support 
staff into regular classes or reguiring the regular academic teacher to provide AIS). The wealth 
and size variables are generahy not significant, though the measure of property wealth (1.01) 
may suggest an increased prevalence of offering AIS during the regular academic class time in 
communities with greater property wealth. NY C and the Big Four are several times more likely 
than rural districts to provide instruction during regular class time. No difference exists 
between rural and suburban districts. 

With regard to which districts provide AIS services in addition to regular academic 
classes, the results are guite clear. 0 nee controlhng for performance and wealth variables, 
urban districts of ah types are less likdy to provide AIS in addition to regular academic classes. 

In fact, NY C, the Big Four, and smah cities are only 8%, 2% and 39% as likely, respectively, as 
rural districts to provide AIS in addition to regular academic classes. There is no significant 
difference between suburban and rural districts. Of note, for both Enghsh (significant) and 
math (only approaching significance) and controlhng for urbanicity, districts serving greater 
proportions of poor children are associated with a practice of providing AIS in addition to 
regular academic classes. 

It is important to note that the aforementioned strategies are not necessarily mutually 
exclusive. It is plausible, and further analyses whl tease this out, that some districts offer AIS 
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before school, during regularly scheduled academic classes, and also in addition to regular 
academic classes. Other districts may hmit their AIS programming to one scheduhng model. 

Instruction 

Turning to the measures of classroom instruction (See Table 10), we sought to 
document general classroom organization and pedagogical strategies, and then determine 
whether differences exist between teacher types (e.g., Enghsh vs. mathematics, responsible for 
AIS vs. not responsible for AIS) and across district (e.g., wealth, location) and classroom (e.g., 
class size, proportion of AIS students in the class) contexts. While we reported on many 
measures above in the descriptive sections, here we focus on three important sets of indicators 
for instruction. 

How does the percent of classroom time spent on whole class instruction, 
with individual students, and in maintaining order vary by context and teacher type 
and responsibilities? 

These measures are important signals of how teachers set priorities for classroom time 
and activities. T eachers in districts that had higher levels of performance in 1999 now tend to 
use more classroom time for whole class instruction than do districts with lower past 
performance, though districts in wealthier communities tend to use the strategy for less of each 
class period than those in poorer communities. The property wealth of the community is 
negatively related to time spent maintaining order in classrooms and positively related to time 
spent instructing individuals. N o differences between urbanicity were found along these 
measures. 

With regard to instructional differences between teachers responsible for AIS 
instruction and those not responsible (contra Ihng for performance, wealth, and urbanicity), 
AIS teachers are one and one- half as likely to spend a greater proportion of class time on 
individual instruction than the non- AIS teachers. Not surprisingly, larger class sizes are 
positively associated with whole class instruction, and negatively associated with individual 
and small group instruction. Moreover, as the proportion of AIS students in a class increases 
(from 0% to 100%), the teachers are twice as likely to instruct in small groups. 

The proportion of class time spent on maintaining order offers a window into 
classrooms and suggests important differences between urbanicity, performance, and high 
concentrations of AIS students in classes. While there are no significant difference in this 
measure between teachers responsible for AIS and those reporting no AIS responsibility for 
the class, teachers instructing greater concentrations of AIS students in the class are more than 
four times as likely to report greater proportions of class time maintaining order. Above and 
beyond this finding is a clear distinction in classroom working conditions between Big Five 
and non-Big Five teachers. It is a near certainty that Big Five teachers report spending greater 
proportions of class time maintaining order than teachers working outside the Big Five. 

Finally, both prior levels of academic performance and gains in achievement over the past four 
years are strongly and negatively associated with time spent on maintaining order. This clearly 
suggests that more orderly classrooms are both related to absolute levels of district 
performance, and to districts that have substantially improved their performance over time, 
regardless of what performance level they started. 
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How do differences in how often teachers lead discussion in class versus how 
often students lead discussions in class vary hy context and type of teacher? 

AIS teachers use student led- discussions more frequently than non-AIS teachers. The 
same is true for Enghsh teachers when compared with mathematics teachers. Similar to the 
finding above regarding whole class instruction, teachers in previously high performing 
districts are more likely to engage in teacher led discussions. In terms of districts making greater 
gains in performance over time, teachers in these improving districts are less hkely to engage in 
teacher led discussions. T eachers in Big Five districts are more than ten times as likely to 
engage in teacher led discussions in class, when compared with their rural colleagues. 

Finally, the teaching of test- taking strategies appears to be strongly related to the 
classroom practice of teachers responsible for AIS instruction (see T able 10). T eachers with 
AIS responsibilities for a particular class are more than three times as hkely to teach test-taking 
strategies. 

H ow does planning among teachers vary by perfomiance and context? 

We examine the frequency of teacher planning in three realms. We first explore 
planning within a department, then across grade levels, and finally between school buildings. 
The data reveal no relation of district performance to reported levels of planning by teachers of 
any type. D espite other findings related to the chahenging working conditions of teachers in 
the largest school districts, we find that Big Five teachers are most likely to report collaborative 
planning in their departments, though the planning 'advantage' does not hold when examining 
planning across grade levels or across buildings. In fact, teachers in the smaher urban and 
suburban districts are more likely to report cohaborative planning across grade levels. Enghsh 
teachers, when compared to mathematics teachers, are more hkely to cohalDoratively plan across 
grades and buildings. 

With regard to AIS, we find teachers responsible for AIS instruction report more 
frequent planning within departments than do those non-AIS teachers. This is an encouraging 
finding given the importance of integrating AIS work with regular class work for under 
performing students. 

In sum, teachers responsible for AIS instruction are more hkely than their coheagues 
without AIS responsibilities to teach test-taking strategies, instruct individual students, and 
engage in student led class discussions. Having a high concentration of AIS students in 
classrooms is strongly related to smah group instruction and greater proportions of class time 
spent on maintaining order. With regard to cohaborative planning, we find a positive 
relationship between AIS teachers and such planning within academic departments when 
compared with their non-AIS teacher coheagues. 

Discussion and Conclusion 

This case from New Y ork State addresses the question of whether state activism can 
focus local district pohcy and program in a manner that increases instructional resources on 
under-performing students. With some caveats, the AIS pohcy mandate has encouraged local 
districts to adhere to guidelines promoting near universal pohcy response of providing 
supplemental instruction for eligible students. While the majority of states have established 
curriculum frameworks and linked them to assessment instruments, this experience in NY may 
be unique for its coordinated emphasis on intervention services (academic and non-academic) 
linked to rigorous learning and accountabihty standards. However, the caveats identified in this 
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study promote a familiar sense of local discretion in the interpretation and implementation of 
state policy mandates. While the primaiy focus in the prior sections entailed a detailed 
description of AIS programming across NY S, including its structural elements as well as 
instructional interventions, in this section we review whether the implementation practices 
were coherent in the development of new instructional capacity and in adhering to state 
regulation. Prior to addressing whether AIS leads under-performing children to achieve high 
standards, we ask whether the pohcy itself is coherent enough to generate appropriate 
instructional capacity to motivate such learning. 

Coherence and Adherence 

We found, generally, that current AIS practices within districts adhere to the general 
criteria estabhshed in the Commissioner's Regulations of NY S, and as further dehneated by the 
Administrative G uidehnes disseminated by the State E ducation D epartment. T able 1 1 
summarizes the degree to which local practice is adhering to state policy. Overall this 
discussion identifies two rather interesting findings. First, implied in the discussion is the 
observation that nearly all schools can demonstrate a high degree of comphance with the AIS 
guidehnes that were enacted in the fall of 2000. This diffusion of practice has appeared rather 
quickly, which contrasts with frequent critiques of pubhc schools as stable and slow to change 
(Cuban, 1983; Tyack & Cuban, 1995). Second, the AIS policy has stimulated new 
organizational and instructional arrangements within schools. The result of such change being 
greater individualized instruction for students performing below accepted performance levels. 
T aken together, these two points indicate a great degree of pohcy adherence by schools across 
the State. 

Specificahy, AIS practice across the State indicates comphance with the fohowing 
general criteria: additional instructional attention to children scoring below accepted 
performance levels; AIS instruction provided by certified staff; multiple measures of 
performance used to determine eligibihty; wide variation in scheduling options provided by 
schools. However, there exists some variation in practice around these general criteria that 
indicates the State guidance is sufficiently broad as to ahow for local discretion and 
interpretation. This may offer support what Moss (2004) describes as coherence- throu^-negotiation, 
local interpretation, and adaptation of the state pohcy to meet local needs. 

There are several sahent examples of local interpretation of the AIS guidehnes. For one, 
the State requires that AIS teachers be licensed to teach the AIS subjects they supervise. The 
survey respondents, however, indicate that more than half of all districts rely on special 
education staff to provide AIS instruction. Special educators are often certified in both subject 
area specialties and in special education, but not always. They are generahy skilled at serving 
students through consulting teacher or inclusionaiy instructional models, and thus may be 
interpreted at the local level as best able to provide AIS instruction despite the intent of the 
regulations. At a quick glance, this could be simple non-compliance; however, the tone of the 
pohcy deviation may also signal a simple cah of professional judgment. For instance, the pohcy 
imphes that teachers with additional subject area expertise (certified in the academic subject 
area of concern) are best suited to provide additional AIS instruction, while this local response 
pattern may reflect an interpretation that expertise in special education services is better suited 
for intervention services. This "non-comphance" can be viewed two ways. This might be 
interpreted as local educators rejecting state pohcy, and hence not adhering to a regulation that 
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seems inconsistent with a policy that calls for improving the learning of each student. 
Conversely, viewed through Moss's coherence- through-negotiation lens, such practice may 
reflect valuable local variation in response and policy coherence. Local educators may size up 
the individual needs of students, make adjustments to staffing and programs, and provide 
what they deem as appropriate supplemental instruction to students in need. D etermining 
which of these interpretations is more appropriate is beyond the scope of our data, but we 
suggest any interpretation of pohcy coherence and implementation should include such debate. 

A different indicator of a lack of adherence to the state pohcy is the finding that some 
students eligible for AIS services are not receiving them. A common response among school 
principals fohowing open ended survey guestions was the statement that they wished their AIS 
program could serve ah students that score a level 2 on a state examination. This finding is 
supported by their observation that 5-15% of children eligible to receive services don't, for 
reasons of scheduling constraints, resources, absenteeism, and lack of student interest. Clearly 
the intent of the AIS regulations is that students scoring below an acceptable performance level 
should receive AIS programming. G iven the clear indicator of who is eligible it may seem like a 
clear case of a lack of interest in complying or an inability to comply. 0 ne plausible 
explanation for the lack of strict compliance is that the guidelines state that students scoring 
below the level of proficiency (Level 3 on the comprehensive assessments) are "ehgible" to 
receive AIS programming. In negotiating local response to the AIS mandate, the use of the 
word ehgible may connote an opportunity to negotiate how many level 1 and level 2 students 
will be served in any given district. 0 f course, issues such as meeting the needs of ah ehgible 
students can go beyond interest in compliance, and bump up against capacity constraints of 
any given district. 
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Capacity 

While the intent of the AIS pohcy is to generate enhanced local capacity to attend to 
the needs of undeipeiforaiing students, the evidence also demonstrates how broader 
constructions of district capacity can impede this implementation. While all districts are being 
told they must provide supplemental services for all low-achieving students, the pohcy has a 
dramatic and differential impact within districts. T ake for instance a 200-student high school 
with 25% of level 1 and 2 students. This school must provide supplemental instruction for 50 
students. Large urban high schools with 2000 students and 50% of students below proficiency 
must provide supplemental services for 1,000 students. 

The research literature on the district role in helping build instructional capacity 
highhghts a set of strategies engaged by district that serve their component schools. Many of 
these strategies, such as providing schools with access to data, professional development, 
curriculum and instructional guidance, gualified staff, and fostering relationships with external 
agents and environment (Masseh & G oertz, 2002; Sphlane, 1996), may be unable to address 
some of the constraints we find in high need NY S schools. For instance, the scheduling of 
students into AIS classes appears to be untenable if half of aU students reguire additional 
instruction in multiple subject areas all at the same time. Specifically, districts may be 
constrained in their abihty to lengthen the school day, build more classrooms, and mandate 
that students stay after school or attend Saturday classes. 

AIS policy was designed to bolster local instructional capacity by demanding the 
addition of supplemental instruction for students in need. This capacity is a key feature or 
nexus between the standard's based curriculum and assessment system. In this sense the State's 
AIS pohcy has enhanced local capacity. 

However, as we have stated, this capacity varies dramaticahy as does the reguired 
response. Elsewhere, we document the importance of local context (e.g., location, size, wealth) 
and how contextual factors relate to levels of local capacity in meeting the new state Learning 
and graduation standards (Sipple & Khleen, 2004). We find that students in districts mth a 
greater proportion of poor students are more likely to offer G ED programs to their students, 
have teachers more likely to agree that a G E D program enhances student learning, and more 
likely to have teachers teaching rote test taking skihs. In short, context matters and hence it 
should not come as a surprise that context can constrain school districts' abilities to comply 
with state regulation. This is especially so when the regulation reguires more than a report to be 
written, a core curriculum to be taught, or a set of exams to be administered. When a state 
demands increase in the amount of instruction, the district response is particularly constrained 
and the challenges are great. 

Of course, the differential impact of state policy has the potential to raise significant 
eguity concerns. This pohcy in NY State has accomplished much and time wih tell how well 
the creation of mechanisms to generate supplemental instructional models in grades K- 12 can 
reduce the need for remediation in later grades. However, it is also clear that the 
disproportionate impact of the pohcy on certain high need schools and districts create 
demands on students' and teachers' time that seemingly cannot be met mth the current six 
hour school day, and the other demands on students schedules. 
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